Bayesian Optimization Algorithm
نویسندگان
چکیده
There are four primary goals of this dissertation. First, design a competent optimization algorithm capable of learning and exploiting appropriate problem decomposition by sampling and evaluating candidate solutions. Second, extend the proposed algorithm to enable the use of hierarchical decomposition as opposed to decomposition on only a single level. Third, design a class of difficult hierarchical problems that can be used to test the algorithms that attempt to exploit hierarchical decomposition. Fourth, test the developed algorithms on the designed class of problems and several real-world applications. The dissertation proposes the Bayesian optimization algorithm (BOA), which uses Bayesian networks to model the promising solutions found so far and sample new candidate solutions. BOA is theoretically and empirically shown to be capable of both learning a proper decomposition of the problem and exploiting the learned decomposition to ensure robust and scalable search for the optimum across a wide range of problems. The dissertation then identifies important features that must be incorporated into the basic BOA to solve problems that are not decomposable on a single level, but that can still be solved by decomposition over multiple levels of difficulty. Hierarchical BOA extends BOA by incorporating those features for robust and scalable optimization of hierarchically decomposable problems. A class of problems called hierarchical traps is then proposed to test the ability of optimizers to learn and exploit hierarchical decomposition. Hierarchical BOA passes the test and is shown to solve hierarchical traps and other hierarchical problems in a scalable manner. Finally, the dissertation applies hierarchical BOA to two important classes of problems of statistical physics and artificial intelligence—Ising spin-glass systems and maximum satisfiability. Experiments show that even without requiring any prior problem-specific knowledge about the structure of the problem at hand or its properties, hierarchical BOA is capable of achieving comparable or better performance than other state-of-the-art methods specializing in solving the examined classes of problems.
منابع مشابه
Comparative Analysis of Machine Learning Algorithms with Optimization Purposes
The field of optimization and machine learning are increasingly interplayed and optimization in different problems leads to the use of machine learning approaches. Machine learning algorithms work in reasonable computational time for specific classes of problems and have important role in extracting knowledge from large amount of data. In this paper, a methodology has been employed to opt...
متن کامل Structure Learning in Bayesian Networks Using Asexual Reproduction Optimization
A new structure learning approach for Bayesian networks (BNs) based on asexual reproduction optimization (ARO) is proposed in this letter. ARO can be essentially considered as an evolutionary based algorithm that mathematically models the budding mechanism of asexual reproduction. In ARO, a parent produces a bud through a reproduction operator; thereafter the parent and its bud compete to survi...
متن کاملProject Portfolio Risk Response Selection Using Bayesian Belief Networks
Risk identification, impact assessment, and response planning constitute three building blocks of project risk management. Correspondingly, three types of interactions could be envisioned between risks, between impacts of several risks on a portfolio component, and between several responses. While the interdependency of risks is a well-recognized issue, the other two types of interactions remai...
متن کاملA Soft-Input Soft-Output Target Detection Algorithm for Passive Radar
Abstract: This paper proposes a novel scheme for multi-static passive radar processing, based on soft-input soft-output processing and Bayesian sparse estimation. In this scheme, each receiver estimates the probability of target presence based on its received signal and the prior information received from a central processor. The resulting posterior target probabilities are transmitted to the c...
متن کاملChoosing Search Algorithms in Bayesian Optimization Algorithm
The Bayesian Optimization Algorithm (BOA) is an algorithm based on the estimation of distributions. It uses techniques from modeling data by Bayesian networks to estimating the joint distribution of promising solutions. To obtain the structure of Bayesian network, different search algorithms can be used. The key point that BOA addresses is whether the constructed Bayesian network could generate...
متن کاملParallelization of Decision Graph Bayesian Optimization Algorithm
The traditional Bayesian optimization algorithm (BOA) is used to generate optimal solutions in Bayesian networks. To enhance performance of BOA, Decision Graphs were introduced which are helpful in parameter (variable) saving. When number of nodes in Bayesian network increases, execution time for computing optimal solution also increases proportionally. So, this paper proposes Parallelization o...
متن کامل